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The major disadvantage of the current Learning Management Systems is the 
lack of learner assistance in their learning processes and, therefore, they can 
not replace the presence of the teacher who ensures the progress of learning. 
In fact, we proposed to integrate, for each learner, a metacognitive agent that 
supported the metacognitive assistance and extracts the defectsin the learning 
process and strategies. The goal is to invite the learner to correct himself and 


Keyword: improve his learning method. Metacognitive questionnaires were distributed 
to a group of 100 students before, during and after a computer course. The 
Agent goal is to evaluatethe metacognitive attributes and to determine their 
Decision tree influence on the success of learning. Decision trees were used as data 
Learning analysis tools to extract a set of rules and to discover the influence of these 
LMS metacognitive attributes on the result obtained by the learners. The results 
Metacognition indicate that there are relationships between the different metacognitive 
attributes and the learners’ success. We note there is the influence of 
metacognitive incitement on learner outcomes, which reflects the degree of 

understanding of a learning pedagogical unit by the learner. 
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1. INTRODUCTION 

Flavell (1976) refers metacognition to a person’s knowledge about his or her own cognition and 
about thecontrol he or she has over it [1]. On the other hand, Brown (1978) refermetacognitive skills to the 
planning, control,monitoring, and self-regulation activities that take place when learning and 
solvingproblems[2], [3]. Several studies indicate thatthe students’ metacognition awarness is a widely used 
instructional strategy to enhance student learning [4]-[7]. Several works have tried to find the relationship 
between the components of metacognition skills, and the motivation or success of learning [8], [9]. Despite 
of the benefits of metacognition skills, its integration in learning management system can be a complex 
problem. In fact,several works have tried to integrate metacognitive skills into distance learning management 
platforms [10]-[13]. Raja M. (2016) used a conversational agents (chatbots) in Open Learner Models (OLM) 
to enhance learner model accuracy and provideopportunities for learner reflection [14]. Themistoklis (2012) 
propose the design of intelligent agent to support teachers in supervising and evaluating learners and 
activities [15]. 

In this paper, we propose to integrate an agent who encourages the learner to make a good planning, 
self-assessment, self-regulation and to determine the appropriate learning strategy by a dynamic dialogue 
system. In order to retrieve data on metacognitive skills and learner perception, we used the forms in a 
dynamic dialogue system from the beginning to the end of reading a learning pedagogic unit. The agent 
receives real-time feedback from the learner via these forms and records the answers for a later analysis by 
means of decision tree. This analysis allows us to find the rules that linked metacognitive skills to the state 
(disoriented, confused and imprudent) of a learner. 
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The absence of the teacher requires the learner to cope with the learning process without a guide and 
orientation. To support and guide learners to succeed learning and avoid the high dropout rate observed [16], 
in this study, the agent triggers a series of incentives before, during and after reading a course. These 
incentives are done at predefined times or automatically based on the rules and learner perception. We find 
that the result of the learners who provide metacognitive assistance is better. 

The paper is organized as follows. Section 2 covers metacognitive skills. Section 3 describes the 
Multi Agents systems characteristics. In Section 4, we present the assessment and data analysis methodology. 
Next, in Section 5 we present result and discution in a real case study. In Section 6, we present our 
conclusions and perspectives. 


2. RELATED WORKS 

Encouraging learners to think in metacognitive terms should have a positive impact on learning 
performance, which is confirmed by the results obtained in 2007 by Maria Bannert [11]. In 2009, a 
relationship was found between self-assessment on the one hand and the effort and performance of learners 
on the other hand by Izaskun Ibabe [5]. In 2012, Nilgiin and his team studied the effect of integrated 
computer-assisted learning on metacognitive incentives on students’ emotional skills [8]. In 2013, Long and 
Aleven argued that self-evaluation and study choice are two important metacognitive processes involved in 
self-regulated learning [17]. 

The relationship between executive functions (inhibitory control and memory work) and 
metacognitive skills was studied in 2014 by Donna Bryce [18]. In 2017, Christina Juliane said that 
motivation is undeniable for the success of e-learning processes [19]. In the same year, a relationship was 
found between learning styles and learner behavior in online learning by Baharudin and his team [20]. 
Indeed, metacognition can take many forms. It includes knowledge of when and how to use learning 
strategies [21]. All the work that has been done shows the impact of a metacognition skill on the learners’ 
performance or the success of the training. However, no work addresses the relationship between 
metacognitive skills and learner status. 

In this article, we determine the possible relationships between metacognitive skills and learner 
states. Indeed, we present an agent model that can motivate a learner to use skills based on his state. The 
agent uses a set of rules and actions to develop the metacognitive skills of the learners. The rules are 
extracted using the decision tree. 


3. METACOGNITION SKILLS 
The position in this paper is consistent with thewidely-held view that metacognition refers to the 
knowledge, control and awareness that students possess in relation to their thinking and learning processes 
[22]. According to Flavell [22], metacognition is defined as the ability to reflect on and control one's own 
cognitive processes which includes knowledge of the 'why', the ‘how' and the 'when’', the learner thus 
engaging in various cognitive activities. Metacognition has the following three main phases: 
a. the planning phase: allows the learner to organize how he or she will use the information, ie define hisor 
her goals, ask himself questions before reading a lesson; 
b. the control phase: allows the learner to make decisions that aim to control the degree of comprehension 
and manage the learning process, ie focus attention, evaluate during reading, maintain motivation, etc .; 
c. the self-regulation phase: allows the learner to focus on activities that are strongly related to control such 
as decreasing reading speed, changing the learning strategy adopted to adjust to the difficulty of the 
course, etc. 


4. THE METACOGNITIVE AGENT 

Agents are software or hardware elements that operate within an environment, act and sense, 
andcommunicate and collaborate with other elements [23]. Every agent follows goals or tasks which are 
specified in the next section [24]. 


4.1. The agents’ roles 

In this study, the agent deals with metacognitive assistance. It reacts according to learners’ feedback 
and association rules extracted by an analysis of data that we have previously collected from students. In fact, 
the agent stimulates learners who need help and offers them metacognitive support in their learning that 
shows in Table 1. 
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Table 1. The Stimulation Performed by the Agent at each Learning Phase [25] 


Learning Phases Stimulation 

Planning Establish the purpose, time and strategy of learning 
Control Control thet learning assessment 

Regulation Adjust the learning plan and strategy 


In this paper, we propose a model of metacognitive agent that collects feedbacks [26], [27] 
(questionnaire, interactions) of learners to measure and evaluate their experiences and their metacognitive 
skills. In this respect, we offer learners a space to take notes, keywords and difficult concepts, as well as we 
require a formative evaluation on the educational support content. We record all their actions and answers in 
order to analyze them by the agent. It aims to determine how and when the agent can intervene to encourage 
learners to improve their learning method that shows in Figure 1. 


UC: Metacognitive agent 


Figure 1. Use case diagram of the metacognitive agent 


4.2. The agent’s activity diagram 

The following diagram shows the triggers of the metacognitive agent in each metacognitive phase at 
a specific time in a learning environment.The agent uses the forms in a dynamic dialogue system from the 
beginning to the end of reading a learning pedagogical unit. The agent receives real-time feedback from the 
learner via these forms and records the answers for a later analysis by means of decision tree. This analysis 
allows us to find the rules that linked metacognitive skills to the state (disoriented, confused and imprudent) 
of a learner. The agent record all learner’s actions and answers in order to analyze them. It aims to determine 
how and when the agent can intervene to encourage learners to improve their learning method that shows in 
Figure 2. In the case where the learner has learning difficulties or has a bad grade, the agent encourages him 
to use the metacognitive skills. 
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Figure 2. Activity diagram of metacognitive incentives and learning environment 


METHODOLOGY 
The variables are divided into a set of metacognitive attributes. The purpose of this work is to 


examine the effect of these metacognitive attributes on the quality of distance learning and on learner 
autonomy in order to stimulate the learner in online learning situations. In this section, we will present the 
method of evaluation of the following metacognitive variables: 


a. 
b. strategies: assess the learner's metacognitive knowledge, 

c. 

d. self-regulation: assess the learner's ability to change the strategy in the case of learning difficulty. 


planning: evaluate the degree of organization of the learner, 
self-assessment: assess whether the learner controls the method adopted in his / her learning ; 


The documentation used to collect data conists of a pedagogical unit and questionnaire. We 


integrate in each unit, which has one or more objectives, questions (at the beginning, in progress and at the 
end) on the metacognitive variables that shows in Figure 3. After reading pedagogical unit and filling in the 
questionnair we distribute evaluation test on the content of the pedagogical unit and the perception of the 
learners. The data was collected from a group of 100 students from the Faculty of Science and Technology of 


Settat (Morocco, North Africa). 
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Figure 3. Structure of documents used to collect data 


Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 3359 — 3367 


Int J Elec & Comp Eng ISSN: 2088-8708 O 3363 


5.1. Assessmentof metacognitive variables 

To evaluate each metacognitive variable, we used a set of questions that have two possible answers, 
“yes” or “no”. Each answer has an elementary note "1" or "-1" that will be used to determine the overall 
score. The overall score is calculated from the sum of the elementary scores divided by the number of 
questions. Thus, the overall score will be between "-1" and "1". In our study, we were satisfied with giving a 
"yes" value to a metacognitive variable if the overall score is greater than zero, otherwise it is assigned the 
value "no". For the evaluation of the planning procedure, the following formula is used: 


planning = if (total (planning) <0): no: yes 


The same formula is used for all other characteristics, namely "strategy", "self-evaluation", "self- 
regulation ", etc. The data relating to the variables are collected via the questionnaire whose structure and 
nature are summarized in the Table 2. 


Table 2. Structure and Nature of the Question 


Planification Strategy Self-evaluation Self-regulation Perception 
1- Goals 1- Use of questioning to 1- Cours 1- Change of 1 Imprudence 
2- Choice of the place, use of understand, memorizing understanding learning method 2-Disorientation 
time, concentration the key terms 2- Identification 2- Use of help 3-Confusion 
3-Interest for the course structure 2- Identification of the of the no 3-Adaptation to 4-Success 
4-Use of tables, diagrams, etc. main ideas and the understand the requirements 
preparation of summary concepts of the course. 


3-Relationships with 
other courses and other 
concepts 

4-Use of "backtracking" 
in case of non- 
understanding 
5-Transition from 
"easy" to "difficult", 
from "simple" to 
"complicated" 


5.2. The association rules between metacognitive variables and the learners perception 

The decision tree that shows in Figure 5, generated from the data collected, shows that there is a 
great deal of dependence between the use of metacognitive strategies (planning, self-evaluation, self- 
regulation and learning strategies) by a learner and his or her perception state (disoriented, confused and 
imprudent). Indeed, we have extracted the following rules: 
a. if not (planning), then the learner is disoriented;(1) 
b. if not (self-evaluation) and not (self-regulation), then the learner is confused;(2) 
c. if not (learning strategies), then the learner is imprudent. (3) 
d. if not (planning) and not(self-regulation) and not (memorization strategies) then unsuccess (4) 

We used the previous rules to define the actions to be taken by the metacognitive agent in the case 
where a learner has a problem during his learning process that shows in Table 3. 


Table 3. Association between States and Actions to be Performed by the Metacognitive Agent 


State Related incentives 
Disoriented Encourage planning 
Confused Encourage the use of self-assessment strategies and self-regulation strategies 
Imprudent Encourage the use of metacognitive strategies, and links between concepts and 


teaching units 


unsucssess Encourages the learner to planning, self-regulation and memorization strategies 


5.3. Integration of the learners metacognitive help 

We assume that pedagogical support is an orderly sequence of pedagogical units and each learner 
must validate previous unit to learn the next one that shows in Figure 4. Otherwise, the agent performs the 
learner's metacognitive (planning, metacognition strategies, self-evaluation and self-regulation) incentive 
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according to learner’s perception (confused, imprudent and disorderly). The agent encourages learners whom 
have the learning disabilities to use metacognitive strategies. 


Validated? Validated? Validated? 


Yes 


Figure 4. Structure of learning support 


5.4. Notation 

(Ux)k=1,..n : a sequence of pedagogical units which composes a pedagogical support S. Each 
pedagogical unit u, is characterized by its pedagogical goals, the minimum note ną to validate and the 
consultation time t, and the tolerance o predefined by the teacher. (ai)i-1..m : is a community of learners 
concerned by the consultation of the same pedagogical medium S. 

TetN;, : the consultation time of pedagogical support and the score obtained by learner a; in the pedagogical 

unit Ux. 

In the case where the learner has not invested sufficient time (Tj,<t,-o) and consequently less effort, 
the agent evaluates the motivation and attention of the learner based on the answers to questions. If the level 
of motivation is low, the agent triggers incentives to motivate and make the learner more active and cautious. 
Otherwise, the agent encourages the learner to review the goals and reschedule the learning. 

In the case where (Tj,>t,+o) and the learner doesn’t leave the learning - in other words, the learner 
finds difficulties or loses motivation - the agent asks the learner: "do you find difficulties? ".Depending on 
the learner's response, the agent will trigger incentives to motivate and make the learner more active and 
cautious if the answer is negative. Else, the metacognitive agent perceives the state S4 of the learner a; at a 
time t.Sti can have the following values: confused, disoriented or imprudent. The agent associates each state 
with an Act, action (incentive associated with the S,; state in Table 3). In the case where the consultation time 
is in the standards (|Tix- ty{<o): 

a. if the learner has sufficient confidence in himself / herself, he / she may pass the assessment test to 
validate the learning unit ux; otherwise the agent encourages him / her to use the self-assessment and self- 
regulation strategies; (rule 2) ; 

b. if the learner has a Ni,rating higher than the validation score ny, he can move on to the next pedagogical 
unit; otherwise, the agent encourages the learner to planning, self-regulation and memorization strategies 
that promote successful learning. These strategies are effectively used by learners who have passed with 
successfullythe unit in question (rule 4). 


6. RESULT AND DISCUSSION 

The decision tree shows that there is a relationship between the learner's state and his metacognitive 
skills that shows in Figure 5. By exploiting the state of the learner, the agent generates interesting 
motivations in natural language, which ‘leads the learner to improve his metacognitive skills. To do this, our 
study focuses, first, on managing the dialogue between the agent and the learners for negotiation and, then, 
focuses on the effects of our agent on the metacognitive skills of learners.We observe that the percentage of 
learners who have successfully completed the learning is greater (71%) with the integration of the 
metacognitive agent than without this agent (30%) that shows in Table 4. Indeed, the result shows the 
importance of the metacognitive agent in the learning process.Indeed, metacognitive incentives allow 
learners to become aware of their mental processes and to improve their learning method. Therefore, the 
learner who has the metacognitive skills easily succeeds in the learning task that shows in Figure 6. 
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Figure 5. Tree induced on real data with a predicted value "state" 


Table 4. The Rules of the Decision Tree 


If planning = no, then the learner is in a messy (disoriented) state. 

If planning = yes and self-regulation = no and self-assessment = no, thenthe learner is in a state of confusion. 

3. If (planning = yes and self-regulation = no and self-assessment = yes) or (planning = yes and self-regulation = yes and 
strategy = no), then the learner is in a state of recklessness. 

4. If (planning = yes and self-regulation = yes and strategy = yes), then the learner is successful. 


Noe 


Result 


30% 71% 


with metacognition without metacognition 


Figure 6. Comparison of learners' results with and without the use of metacognition 


7. CONCLUSION AND PERSPECTIVES 

This study was conducted to examine the effect of different metacognitive variables on the success 
of the learning process. The result obtained shows that the integration of a metacognitive agent into the 
distance learning management system can significantly improve metacognitive skills and knowledge. The 
main goal of our study is to encourage the learner to use metacognitive skills in their online learning process. 
The result shows that the influence of the metacognitive agent on the e-learning process is important because 
it develops the learner's autonomy, maintains his or her intention and motivation,helps him or her to acquire 
metacognitive skills such as planning, self-assessment, self-regulation, techniques and memorization 
strategies. In a future work, we intend to build a standard metacognitive agent model that can communicate 
with the LMSs and base the rules dynamically extract. We will improve its functioning so that it can 
automatically detect the state and profile of the learner and react in real time. 
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